Sensor-based detection of service event condition within a single defined service area by a service robot

ABSTRACT

A method and system for a reconfigurable robotic platform through a plurality of interchangeable service modules and adapted to engage in both autonomous and interactive maintenance and monitoring of a service area, the robotic platform configured to execute a stored service plan for a service area, the service plan comprising a service plan sequence comprising a service treatment to a surface of the service area, identify a special treatment area utilizing the surface sensor, alter the service plan sequence based on identifying the special treatment area in the service area, wherein the altering of the service plan sequence includes providing an additional service to the special treatment area, and resume execution of the service plan.

CLAIM OF PRIORITY

This patent application is a continuation of U.S. patent applicationSer. No. 15/646,551 (DROB-0004-U01) filed on Jul. 11, 2017.

U.S. patent application Ser. No. 15/646,551 claims priority to U.S.Provisional Patent Application Ser. No. 62/361,557 (DROB-0001-P01) filedon Jul. 13, 2016.

U.S. patent application Ser. No. 15/646,551 is a continuation-in-part ofU.S. patent application Ser. No. 14/590,597 (DROB-0003-U01) filed onJan. 6, 2015, now granted as U.S. Pat. No. US9,722,640.

The entire contents of each of these U.S. Patent Applications are herebyincorporated by reference in their entirety.

FIELD

The present application generally relates to a robotic platform. Inparticular, the present application relates to apparatus and methods forproviding a reconfigurable robotic platform with interchangeable servicemodules adapted to engage in both autonomous and interactive maintenanceand surveillance of generally planar environments.

BACKGROUND

While robotic systems are in use that perform simple cleaning andsurveillance, such robotic systems are designed for specific andwell-defined purposes. The high degree of specificity in each system'sdesign and function results in expensive robotic systems that aretailored to perform only very specific tasks.

What is needed is a single robotic platform that may be configured toperform a wide variety tasks.

SUMMARY

The present disclosure describes a method and system for areconfigurable robotic platform utilizing a plurality of interchangeableservice modules and adapted to engage in both autonomous and interactivemaintenance and monitoring of a service area, the robotic platformconfigured to perform a wide variety of tasks utilizing the plurality ofinterchangeable service modules, and navigating through the service areautilizing a plurality of sensors and guided through a stored serviceplan for the service area.

In embodiments, a robot may comprise a propulsion mechanism, apositioning mechanism, and a removable service module adapted to performa predefined service task. In embodiments, the positioning mechanism maycomprise a LIDAR system, stereoscopic imaging system, RF locationfacility, and the like. The removable service module may comprise apower source. The removable service module may comprise at least onesensor. The at least one sensor may be an imaging component adapted tosense a condition of a generally planar surface proximate the robot. Theat least one sensor may comprise a camera. The at least one sensor maycomprise a LIDAR system. The predefined service task may comprise theservicing of a generally planar surface proximate the robot. Thepositioning mechanism may comprise a stereoscopic camera system. Thepredefined service task may comprise a single service task forming apart of a service plan comprising a plurality of service tasks where theplurality of service tasks are directed to a plurality of non-contiguousgenerally planar surfaces, where at least two of the plurality ofnon-contiguous generally planar surfaces may be separated by elevation.

In embodiments, a method may comprise engaging a propulsion mechanism ofa robot comprising a removable service module to position the robot at apredetermined location, and operating the removable service module toservice a generally planar surface proximate the robot. Positioning therobot may comprise sensing an environment around the robot, anddetermining a position of the robot by comparing the sensed environmentto a model of an environment. Sensing the environment may compriseutilizing a LIDAR system. Sensing the environment may comprise utilizinga stereoscopic image. The method may further comprise selecting aremovable service module for insertion into the robot. Selecting theremovable service module may comprise selecting the removable servicemodule based, at least in part, on a service plan. The service plan maycomprise a plurality of areas each with an associated defined removableservice module for servicing each of the plurality of areas. The methodmay further comprise changing the removable service module in responseto a change in a type of the generally planar surface proximate therobot. Changing the removable service module may comprise sliding theremovable service module into the robot along at least two railsattached to the robot.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe substantially similar components throughout the severalviews. Like numerals having different letter suffixes may representdifferent instances of substantially similar components. The drawingsillustrate generally, by way of example, but not by way of limitation, adetailed description of certain embodiments discussed in the presentdocument.

FIG. 1 illustrates operational functions and operational environment ofa robotic platform according to exemplary and non-limiting embodiments;

FIG. 2 illustrates perspective renderings of a robotic platformaccording to exemplary and non-limiting embodiments;

FIG. 3 illustrates a perspective rendering of a robotic platform withremovable service module according to an exemplary and non-limitingembodiment;

FIG. 4 illustrates a functional block diagram of the components of arobotic platform and service module according to an exemplary andnon-limiting embodiment;

FIG. 5 illustrates a functional block diagram of the components of therobotic platform relating to a process of robot localization in anexemplary and non-limiting embodiment;

FIG. 6 illustrates a multi-sensor pose estimation process according toan exemplary and non-limiting embodiment;

FIG. 7 illustrates a functional block diagram of the components of therobotic platform relating to a process of sensing, obstacle avoidance,and path planning in an exemplary and non-limiting embodiment;

FIG. 8 illustrates a service module comprised of a cleaning system inaccordance with an exemplary and non-limiting embodiment;

FIGS. 9-33 illustrate graphical user interface display content inaccordance with exemplary and non-limiting embodiments.

DETAILED DESCRIPTION

FIG. 1 depicts a robotic platform 100 configured to perform a widevariety of tasks, such as in the servicing of multiple surfacesutilizing multiple service modules. Although the description hereinpertains predominately to an embodiment for providing cleaning servicesin an indoor space (e.g., cleaning floors), it should be understood thatthe robotic platform 100 may be adapted to service a plurality of otherenvironments, such as servicing outdoor traffic surfaces (e.g., cleaningroads and walkways), outdoor groomed landscapes (e.g., mowing lawns),industrial faculties (e.g., warehouses, truck loading areas,manufacturing facilities), and the like. Further, the cleaning of indoorspaces as described herein, although described generally in terms ofopen interior floor, it should be understood that the robotic platform100 may operate in any area through which it can physically operate(e.g., small rooms, hallways, walkways, and the like). One skilled inthe art will also appreciate that the size and function of the roboticplatform 100 may be scaled to accommodate various applicationenvironments. For instance, a robotic platform 100 configured forcleaning large commercial spaces, such as an airport terminal, may belarge, with service modules utilizing correspondingly high capacitycleaning reservoirs (e.g., cleaning fluid) and resources (e.g., cleaningtool head). The robotic platform 100 may also be configured forresidential applications, where the physical dimensions of the roboticplatform 100 are sized appropriately to cleaning within the confines ofa personal residence, utilizing correspondingly sized service modules,reservoirs, and cleaning head. Alternately, in an outdoor environment,where physical constraints are reduced, the robotic platform 100 may beconfigured to be larger (e.g., wider, to maximize coverage).

FIG. 1 depicts operational functions and operational environment of therobotic platform 100 according to exemplary and non-limitingembodiments, and is not meant to be exhaustive, but rather illustrativeof particular operational features of the robotic platform. As depictedin FIG. 1, the robotic platform 100 includes a main body 214 thataccommodates one of a plurality of service modules (102A-D), along withprocessing 106 capability as informed through a variety of navigationaland resource sensors 104. Further detail of the robotic platform 100 isprovided herein, but a key feature of the robotic platform is itsability to accommodate different service modules 102A-D, each of whichmay provide a different functional capability. For example, in an indoorfloor cleaning application one module may be for vacuuming, another forrug washing, and still another for floor waxing. These specific examplesare meant for illustration purposes only, and do not imply that theseare functional boundaries between various service modules 102A-D.Rather, the ability of the robotic platform to accommodate a variety ofservice modules represents a benefit of the modular capability of theplatform, where each service module may be optimized for its functionrather than having some compromised subset of functions constrainedwithin a single fixed functionality unit (e.g., a ‘one-does-all’ robot),or requiring separate individual units for each service capability(e.g., one robot for cleaning, another robot for vacuuming, and stillanother robot for waxing). Modularity enables the robotic platform 100to optimally provide a wide variety of service functions with a singlerobotic platform.

The robotic platform 100, through a main body 214 portion, providescommon functions and processes for executing service tasks for theplurality of service modules 102A-D. The robotic platform 100 providesprocessing 106 capabilities (e.g., electronics, processing, software,and the like) to execute operational functionality. In embodiments,processing 106 may be provided in a distributed and/or shared-processingarchitecture between the main body 214 of the robotic platform 100 andthe service modules 102A-D. For instance, the main processingcapabilities for navigation and service plan execution may bepredominately located in the main body 214 of the robotic platform 100,but some software, such as for motor drivers, and the like, may beunique to the module (e.g., located in the memory of the main body 214of the robotic platform 100 or in memory of the service module 102A-Dprocessing. In embodiments, there may be multiple levels of processing106, such as a high-level processing module responsible for overallmanagement of the robotic platform 100, an operational level processingmodule that takes navigation sensor input and collects and sendsinformation up to high level processor for managing control, and finallya service module level processing module, where processing may be sharedthrough the multiple levels (e.g., through shared processing between themain body processing and the service module processing). In embodiments,processing capabilities in the main body 214, such as at the high-levelprocessing module, may be configured to be customizable for a pluralityof service module level processing modules. For instance, each servicemodule 102A-D may have a unique identifier, and when the service module102A-D is mated with the main body 214 the processing functionality inthe main body may identify a module type, a tool type, a softwareversion, and the like, by the unique identifier, and determine an actionassociated with this particular service module. For example, a servicemodule 102A may be a new version of a mopping service module, withupdated software and/or hardware components. The processingfunctionality in the main body 214 may then have the capability toaccommodate the updated mopping service module through customizablefunctions and messaging. In embodiments, software for one or more of theprocessing levels may be updateable, such as through wireless automaticupdates. For example, the software for top-level processing 106 may beupdateable, such as to accommodate updates to processing in the mainbody or for upgrades to service modules, where the top-level processingthen appropriately modifies operations and communications with the lowerlevels of processing per the software updates.

Processing 106 functionality interfaces with sensors 104, such asutilized for navigating (e.g., through imaging, laser ranging, sonar,external RF locating nodes 150A-C), sensing of surface characteristics(e.g., image recognition, surface reflectivity measurements, contactsensors), sensing internal operational parameters (e.g., reservoirlevels, fluid cleanliness quality, performance characteristics), and thelike. Sensors 106 may be located in the main body 214 of the roboticplatform 100 and/or in the service modules 102A-D, such as dependentupon their use and function. For example, navigation sensors may belocated in the main body 214 of the robotic platform 100 because theyprovide a resource that the robotic platform 100 will requireindependent of which service module 102A-D is in use. Alternately, somesensing capabilities may be service module specific, and so locatedwithin the service module. Service module 102A may, for example, requirea sensing capability that service module 102B does not. For instance,service module 102A may be a rug cleaning service module with a cleaningfluid reservoir, where the cleaning fluid reservoir is monitored bysensors for quality (e.g., sensing cleanness quality of the fluid in thereservoir), level (e.g., sensing how much fluid remains), and the like.Service module 102B may be a vacuum service module with no cleaningfluid reservoir, but has a material collection container that ismonitored for fullness. The capability for distributed processing 106and sensors 104 maximizes the flexibility of the robotic platform 100 inthe accommodation of a plurality of service modules 102A-D.

The robotic platform 100 may provide a service planning facility 120,such as including the ability to generate and store service plans 128for one or more task areas. A service plan 128 may utilize a mappingfacility 122, such as with capabilities to generate and utilize digital2D maps 124 and 3D maps 126 for navigating through and providingplanning services for service areas 140A-B. Digital maps may provide for2D-based service area layouts and features, such as dimensions of thearea, surface characteristics, objects permanently present in the area,objects transient through the space, entrance and exit locations forservicing the area, and the like, enabling navigation and planningthrough the service area. Dependent upon the size and complexity of thetask area specified in the service plan 128, the digital map may bequite large. In order to increase computational efficiency, a digitalmap for a task area may be divided up into a plurality of work areas,where operations through the plurality of work areas are coordinated.Dividing up a large task area into a plurality of work areas mayincrease computational efficiency by limiting the size of the digitalmap being presented for processing at any one time. For example, thedigital map for a large task area may be stored in memory of the roboticplatform 100, but where the task area digital map is further dividedinto work areas and loaded into working memory in chunks or as aprogressive load. Maintaining computational efficiency through dividinga task area digital map into a plurality of work area digital maps mayenable the robotic platform 100 to maintain a higher level of spatialresolution processing for navigating and servicing the task area. A taskarea may be divided into different sizes or levels/sub-levels of workarea based on the size of the task area, service treatment for differentwork areas, tools required for different work areas, service modulerequirements, and the like. Digital 3D maps may augment the utility of2D maps by providing 3D-based service area features, such as lowclearance areas, incline angles and locations, vertical transitionlocations between or within service areas (e.g., elevator, ramp), raisedfeatures in the service area (e.g., a bump, cable cover), and the like.In embodiments, the service planning facility 120 may be providedthrough a user interface of the robotic platform 100 or through anexternal computing facility (e.g., the user computing device 130). Inembodiments, a service area mapping 142 of a service area may begenerated by the robotic platform 100, such as through exploring theservice area as part of a service area plan setup, or generated by anexternal mapping facility and downloaded to the robotic platform 100. Adigital map may then be used in determining the movements of the roboticplatform 100 throughout the service area, such as determined by aservice plan 128.

Although service areas described herein often utilize illustrativeembodiments with respect to two-dimensional surfaces (e.g., floors),work areas may exist on multiple levels (e.g., multi-floors of the samebuilding), where each level has its own digital map and work areadescription. The robotic platform 100 may be able to move between levelsusing RF communication means to activate transition mechanisms (e.g.,elevators, lifts) allowing movement between levels without humanintervention. The robotic platform may be able to identify leveltransition points as a means of moving from work area to work area ondifferent levels. Physically the work areas may be on different levels,but the robotic platform 100 may implement an overall service plan asone logical network of interconnected work areas through a limitednumber of access and level transition points. For example, the roboticplatform 100 may finish with a first work area on a first level and thenutilize an elevator that is identified as a level transition point toproceed to a second work area on a second level. In embodiments, therobotic platform 100 may utilize an RF signal to call the elevator,enter the elevator on its own, send an RF signal to select the floor,and then exit the elevator when the proper floor has been reached. Therobotic platform 100 may utilize elevator RF signaling protocols tocommunicate with an elevator system, such as where the elevator systemincludes an RF transceiver for communication with the robotic platform100.

Navigation through service areas 140A-B may utilize a combination ofdigital map usage (e.g., localization determined based on navigationthrough the mapped area) and real-time sensors (e.g., sensors 104monitoring the robotic platform's surrounding environment). In addition,a plurality of RF locator nodes 150A-C may be used to navigate andlocalize, as described herein, where the robotic platform 100 senses thelocation of the RF locator nodes 150A-C, such as in relation to theirlocations stored on the 2D digital map 124. The locations of the RFlocator nodes 150A-C on the 2D digital map 124 may be determined in anumber of ways, such as through direct input of the RF locators 150A-Cinto the map by a user who has determined the locations of the RFlocator nodes 150A-C, the robotic platform 100 may determine thelocations of the RF locator nodes 150A-C during a service plan setupprocess (e.g., determining the locations as the robotic platform isnavigated around the space, either automatically or manually by a user),and the like. In embodiments, the locations of the RF locator nodes150A-C may be determined through a node location facility thatdetermines the location of each of a plurality of RF locator nodes150A-C in an area. The node location facility may utilize a referencepoint transmitter that is placed in the area to determine a first set oflocation coordinates for the plurality of RF locator nodes 150A-C. Thereference point transmitter is then moved to a second location todetermine a second set of location coordinates, and then a thirdlocation to determine a third set of location coordinates. These sets oflocation coordinates may then be used to geometrically determine thelocation of the plurality of RF locator nodes 150A-C. These locationsmay then be input to the 2D-digital map 124 to automatically andaccurately locate the plurality of RF locator nodes 150A-C on the 2Ddigital map for use by the robotic platform 100 in localization withinthe area. This procedure may be repeated for different areas withdifferent sets of RF locator nodes, and represents a system for easilydetermining the locations of RF locator nodes for new areas, or forexisting areas where RF locator nodes have been added (e.g., to cover adead-spot) or moved.

RF locator nodes 150A-C may be located across an extended service area,such as across a large area, in multiple adjacent areas, throughout afacility, and the like, where the robotic platform 100 utilizes the bestset of RF locator nodes 150A-C available to it. For instance, in a largearea such as an arrival or departure area in an airport terminal, theremay be a plurality of RF locator nodes located throughout the area, andas the robotic platform 100 moves through the area it may use the bestset of RF locator nodes 150A-C available, such as the closest nodes, thenodes with the best signal, the set of nodes that provide the bestspatial positioning combination, and the like. As the robotic platform100 moves through the service area it may switch RF locator nodes in acontinuous process of optimization. For instance, when a roboticplatform 100 enters a service area it may sense and utilize nodes 1, 2,and 3, but as it progresses through the area it may get farther fromnode 1 but closer to a new node 4, and at some point (e.g., determinedby signal strength, by geometric layout, and the like) switch from usingnodes 1, 2, and 3 to using nodes 2, 3, and 4, where switching is ahandover from one set of nodes to another set of nodes, where the firstand second set of nodes may or may not comprise an overlapping set ofnodes. Thus, the robotic platform 100 may constantly assess and optimizethe set of RF network nodes 150A-C it uses as it moves through a servicearea or through multiple service areas.

RF network nodes 150A-C may augment the use of sensors 104 utilized forlocalization and navigation through work areas of a service plan.However, RF network nodes 150A-C may not be required, such as where therobotic platform 100 utilizes only on-board sensors 104 for localizationand navigation. In embodiments, some service plans for facilities maynot require RF network nodes 150A-C at all, and as such, no RF networknodes 150A-C may be installed in a service area 140A-B. RF network nodes150A-C are most useful in operational situations where ambiguity mayexist, such as in a very large or long room. As such, RF network nodes150A-C may be selectively employed as required to eliminate suchambiguity. In some cases they may be the only technology that canresolve ambiguity. However, there are a plurality of navigation orlocalization systems available to the robotic platform, such as the RFnetwork node system 150A-C, stereoscopic imaging sensors (e.g., 2DLIDAR, stereo camera), 3D LIDAR, dead reckoning, and the like. Therobotic platform 100 may optimally combine information from any one orcombination of the sensor 104 and RF network node 150A-C facilities. Inembodiments, optimization may be implemented by dynamically weightingthe availability and performance of each of the available sensor 104and/or RF network node 150A-C inputs.

The robotic platform 100 may navigate through the service area through acombination of sensor-based position estimation and positionalpredication based on the physical movements of the robotic platform 100.For instance, positional encoders from the propulsion mechanisms mayprovide odometry data for positional measurements, and based onkinematic algorithms and the plan for movement (e.g., per the serviceplan and 2D digital map), may be used to predict where the roboticplatform 100 will be in time. This prediction may then be compared witha combination of sensor-based position measurements, such as from theLIDAR, camera, IMU, and the like. This comparison may then beprobabilistically analyzed to determine a best estimate for the currentposition of the robotic platform 100. In embodiments, in the absence ofRF node locator signals (e.g., unavailable because of interference orfailure), the robotic platform 100 may utilize this process innavigating through sensor-augmented dead reckoning or other localizationmechanism such as stereoscopic imaging sensors, 3D LIDAR, and the like.

Service plans 128 store the details for providing service to a servicearea 140A-B, such as mapping for navigation around the service area,features of the service area and how to navigate around them, a schedulefor servicing, service modules to be used, consumable resources, orderof service modules, and the like. Service plans 128 may include servicedetails for one or more different service areas. For instance, theservice plan may include a plurality of service areas, such as servicearea 140A and a service area 140B. Service area 140A and service area140B may have different layouts and surface characteristics, such asservice area 140A being an interior area with a linoleum floor andservice area 140B being an indoor-outdoor area with a rug. Each area mayhave different fixed obstacles 146 (e.g., furniture) or may have serviceplan notes for activity within the area, such as the area being a highpedestrian traffic area or where furniture is regularly moved around. Atask area may include different requirements for tracking and treatingaround the periphery of a work area. For instance, a work area may havea hard boundary (e.g., a wall) or a soft boundary such as the boundarybetween two different treatment surface areas (e.g., between a rug and atile floor). Treatment at a boundary may require different operationalconstrains in order to not apply service over the boundary (e.g.,getting a wall or adjacent work area wet from a cleaning treatment),such as slowing down the movement of the robotic platform 100, slowingdown the motion of a tool of a service module 102A-D, applying lesscleaning material to the surface, applying a cleaning material with lesspressure, and the like, as the robotic platform 100 navigates near thetask area boundary. In embodiments, the service plans 128 may beinitially generated during a service set-up, but may also be updated. Inembodiments, a user may provide updates to the service plan 128. Inaddition, the robotic platform 100 may be enabled to update the serviceplan 128 based on new information gathered during regular serviceexecution. For instance, the robotic platform 100 may usemachine-learning to determine changes to service areas 140A-B, andincorporate those changes into a revised service plan (e.g., furniturehas been moved from a location initially identified during setup of theservice plan, to a new area).

In embodiments, the robotic platform 100 may adapt a service plansequence based on conditions encountered during service treatment. Forinstance, the robotic platform 100 may, during the cleaning of servicearea 140B, encounter a special treatment area 144 (e.g., a detectedstain on the floor, a user-marked or spot-treated area on a floor, ahigh level of soiling requiring special treatment). The robotic platform100 may sense the special treatment area (e.g., through surface sensing,imaging) or a user may input the location of the special treatment area144 for special attention. The robotic platform 100 may provideadditional service to the special treatment area 144 (e.g., apply morepressure to the cleaner, slow down the speed of the robotic platform toincrease the amount of cleaner provided, go over the area a second time,and the like) at the time the robotic platform 100 first encounters itor at a later time (e.g., scheduling a return to the area). In anotherinstance, the robotic platform 100 may encounter an obstacle 146 andadaptively go around it, or encounter a special treatment area 144 andadaptively skip the area for later treatment, such as at the end of thecurrent task or with another service module type at a later time. Inembodiments, the robotic platform 100 may store the location of theobstacle 146 or the special treatment area 144 for later treatment.Further, the robotic platform 100 may perform an analysis with respectto postponed treatment areas. The robotic platform 100 may analyze thetime required to return to an area against the benefit of returning tothat location given the time constrains for the day. For instance, theremay be number of areas that have been skipped, and there is not enoughtime to return to all of them, so the robotic platform 100 may performan analysis to maximize the benefit in the time allocated for completionof the area service.

For example, during the course of executing a daily service plan 128,the robotic platform 100 may encounter two areas that need specialtreatment 144 (e.g., one small and one large soiled area) and oneobstacle 146 (e.g., a chair in the middle of the floor) that forced therobotic platform 100 to skip the area around the obstacle. The roboticplatform 100 may have immediately determined that the special treatmentarea identified as a small re-treatment area should be re-treatedimmediately (e.g., because the time required for retreatment was below amaximum time threshold for immediate re-treatment), and as such, appliedtreatment a second time, such as in a repeat pass of the area. However,the second special treatment area was determined to take an amount oftime that exceeded a maximum threshold of time for immediatere-treatment, and whose location was instead stored for possible laterre-treatment. Similarly, the robotic platform determined that theobstacle was a static obstacle and stored its location for possiblelater re-treatment. Once the robotic platform 100 completed the taskarea, or was done for the day (or some other break in the service plan),a return plan may be processed that considers the possibility ofreturning to the stored locations of the special treatment area and theobstacle for re-treatments. In this instance, the robotic platform 100may conclude from the analysis that it should return to the specialtreatment area because it will take an amount of time that is within atime constraint availability (e.g., there's enough time left in the workday to complete the re-treatment, there's enough charge left in thebatteries to complete the re-treatment, and the like). However, althoughthe analysis determines that the static obstacle represents a smallre-treatment area it also determines that probabilistically the staticobstacle will still be in place upon return (e.g., the chair will stillbe there), and in weighing the time available against the probabilitythat the obstacle is still there, determines to not return for aretreatment at this time. As such, the robotic platform 100 notes thearea for extra treatment on a subsequent servicing of that area onanother day. The robotic platform is thus adaptable to changingconditions and is able to store and analyze the need for changes in itsservice plan routine based on environmental changes encountered.

The robotic platform 100 may utilize a service robot resource facility110, such as at a ‘home’ location that provides resources, includingservice module material supply (e.g., consumable materials, cleaningheads), service module exchange 114, electric charging station 116, andthe like. In embodiments, one or more of these functions may beautomatic where the robotic platform 100 executes a function withoutuser support, assisted where the robotic platform 100 performs someaspect of the function but with user support, or manual where the robotplatform 100 returns to the service robot resource facility 110 for userexecution of the function. For functions where a user is required to bepresent, the robotic platform 100 may wirelessly communicate with a usercomputing device 130 to notify the user (e.g., sending alarms and alertsto the user whenever needed to prompt the user to action, inform theuser of a completed action, to change the replaceable service module,and the like). In embodiments, the user computing device 130 may providefor a user interface for communicating with, and monitoring the progressand performance of, the robotic platform 100. For automatic orsemi-automatic functions, the robotic platform 100 may return to theservice robot resource facility 110 and autonomously perform thefunction. For example, the robotic platform may return to the electriccharging station 116 when its batteries are low or at the end of aservice day. The electric charging station 116 may comprise acontactless charging facility that enables the robotic platform 100 toautomatically charge its batteries while in proximity to the chargingfacility. In another example, the robotic platform 100 may automaticallyconnect to a consumable resource port of a service module materialsupply 112, such as a station for filling or emptying fluids.

The robotic platform 100 may service multiple service areas utilizingmultiple service modules, such as in a coordinated process outlined in astored service plan. Therefore, the robotic platform 100 may have acontinuous need to switch between service modules 102A-D. To aid inaccomplishing this exchange, the service module exchange facility 114may be located at a designated location where service modules 102B-D arestored while the robotic platform 100 provides a service with servicemodule 102A. When the robotic platform 100 needs to switch betweenservice modules 102A-D, it may do so through aid of a user orautomatically through the service module exchange facility 114 (e.g., amechanism for automatically switching between service modules). As withthe case of refreshing or replacing consumable resources through theservice module material supply 112, the robotic platform 100 maywirelessly communicate with a user through a user computing device 130.For example, the robot platform 100 may have completed vacuuming a rugcovered service area 140B with service module 102A, and per a serviceplan sequence, is next directed to wash a linoleum covered service area140A with service module 102B. The robotic platform 100 may thenautomatically proceed to a pre-designated location for exchanging theservice modules 102A-B, such as at the service robot resource facility110, and simultaneously communicate a need for a level of user supportto make the exchange through a message sent to the user computing device130. The user may then assist the robotic platform in the exchange towhatever extent is required, such as the user manually exchanging theservice modules, the user aiding a semi-automatic process for exchangingthe service modules, the user monitoring an automatic exchange of theservice modules, and the like.

With reference to FIG. 2, there is illustrated an exemplary andnon-limiting embodiment of a robotic platform 100. As illustrated,robotic platform 100 is adapted to travel over generally planar surfacesvia the operation of a propulsion mechanism 202. Propulsion mechanism202 may include a drivable wheel assembly or other mechanism capable ofproviding controlled motion of robotic platform 100. Robotic platform100 further includes a top mounted imaging system 204 adapted to imagethe surrounding environment. Imaging system 204 may be comprised of a 3DLIDAR system adapted to produce three-dimensional point cloudinformation indicative of a surrounding environment. Such point cloudsmay be comprised of a 360-degree rotational sweep about the roboticplatform 100 whereat, for each degree of sweep, there is incorporateddata comprising imaging in the z-direction. In some embodiments, theimaging system 204 may be comprised of a stereoscopic vision systemadapted to produce a three-dimensional model of the surroundingenvironment. While described herein as incorporating, generally, a 3DLIDAR system, in practice either the 3D LIDAR system or the stereoscopicimaging system may be used alone or in combination.

Robotic platform 100 may further comprise one or more ultrasonic sensors206. Ultrasonic sensors 206 operate, generally, to detect near fieldobjects in the direction of movement of the robotic platform 100 asdescribed more fully herein. Robotic platform 100 may further compriseone or more 2D LIDAR systems 208. Each 2D LIDAR system operates,generally, to image a two-dimensional wedge formed by scanning a laserin front of the robotic platform in the direction of movement, such asforward motion relative to the orientation of the robotic platform,reverse, or to a side during turns. In addition, robotic platform 100may include a camera 210 for providing images using visible light,near-IR, and IR wavelengths. Camera 210 may be adapted to image,generally, but not exclusively, in front of the robotic platform 100 inthe direction of movement.

Robotic platform view 100A illustrates the service module 102 (any of102A-D) mounted inside a main body 214 of the robotic platform 100. Asdescribed more fully herein, service module 102A-D is an interchangeableand self-contained element that may be removed from robotic platform 100and replaced by one or more other service modules 102A-D. FIG. 3 depictsa robotic platform view 100B with the service module 102 removed. Eachservice module 102A-D is but one of a plurality of types of servicemodules each directed to a distinct and defined operating mode,depending on the desired service. In some embodiments, a service module100 may be comprised of its own processor for managing the operation ofthe service module 100 and the various components forming a part of theservice module 102. In some embodiments, a service module 102 mayincorporate its own power storage and/or power generation system forproviding power to itself and/or to the robotic platform 100 of which itforms a part. In addition, each service module 102A-D may incorporateone or more sensors including, but not limited to, pressure sensors,moisture sensors, LIDAR systems, imaging systems, and the like, tailoredto performing in an operating mode for which the service module 102A-Dis designed. Note that in other embodiments, the main body 214 of therobotic platform 100 may be equipped with similar sensing and processingcapabilities to those described in connection with the service module102A-D.

In an example, a service module 102A adapted to provide cleaningservices to a surface comprised of thick rug may incorporate a 2D LIDARsystem for evaluating, either statically or dynamically, atwo-dimensional profile of the rug to determine surface roughness orpile length. The same service module 102A may comprise an imaging systemfor sensing anomalous stains in the rug so as to apply additionalcleaning solution. In other instances, a service module 102B may includea series of sensors and mechanisms designed to buff and polish marblefloors. In each instance, the service module 102A-D provides functionsappropriate to a particular task according to a desired mode ofoperation as inserted into robotic platform 100. As described more fullyherein, the interchangeability of the service modules 102A-D may beachieved via human intervention or without direct human intervention inaccordance with a defined automated operation regiment.

As noted, an imaging system of the robotic platform 100 may incorporatea 3D LIDAR radar system and/or stereoscopic imaging system (e.g., 2DLIDAR or stereo cameras). As described more fully herein, the 3D LIDARand/or stereoscopic imaging system may be used, for example, toestablish a static or semi-static mapping or representation of anenvironment in which the robotic system 100 is intended to operate. Inother exemplary embodiments, the 3D LIDAR system and/or stereoscopicimaging system may be used to dynamically update an existing map, toperform localization, to perform pose estimation, to perform objectand/or obstacle detection, to perform and verify obstacle mitigation,and the like.

In accordance with exemplary and non-limiting embodiments, the 3D LIDARand/or stereoscopic imaging system may be used to create an initialmapping, in 2D or 3D, of an environment in which the robotic platform100 is intended to operate. For example, the robotic platform 100 may beguided along a path or paths within a defined service area 140A-B inwhich it is to operate so as to create a 3D point cloud representing thethree-dimensional structure of the environment. The 3D point cloud soproduced may be algorithmically processed with or without humanintervention to produce a three-dimensional model of the environmentsufficient to permit the operation of the robotic platform 100 inaccordance with a predefined or dynamically determined mode ofoperation. Once created, the three-dimensional model may be storedwithin a memory of the robotic platform 100 or made available from anauxiliary memory such as, for example, wireless communication to anexternal data server. The initial traversing of the environment tocreate the three-dimensional model may be performed autonomously by therobotic platform 100. In other instances, the initial traversing mayincorporate human guidance. For example, a user may use a wirelessjoystick to guide the robotic platform 100 around a prospective servicearea 140A-B while creating a three-dimensional model. In some instances,the creation of such a model may be offered as a service to customers.In other exemplary embodiments, the three-dimensional model may beconstructed by an apparatus not forming a part of the robotic platform100 but otherwise communicated to the robotic platform 100, such asdownloaded to the robotic platform 100 from an external computingfacility.

Regardless of the method by which the three-dimensional model isconstructed, the model may be used to perform localization. As usedherein, “localization” refers to the process of determining a spatiallocation within a predefined environment. As described more fullyherein, localization may make use of a plurality of sensors 104 andinputs at any time. For example, 3D LIDAR and/or stereoscopic imagingsystem may be used in conjunction with ultra-wide band signaling via RFlocator nodes 150A-C, when required, to determine a position of therobotic platform 100. In some embodiments, the robotic platform 100continually scans its environment to produce a localized 3D model of itsenvironment. The robotic platform 100 may then compare this localizedmodel to the predetermined three-dimensional model of the environment,such as via a statistical, best-fit methodology including, but notlimited to, principle component analysis, regression, Kalman filteringand the like, in order to determine a position of the robotic platform100. Such comparing need not be performed blind, but, rather, mayincorporate predictive technology. For example, once the roboticplatform 100 determines its position within an environment and moves asmall distance, subsequent positional scans of the surroundingenvironment may commence to match their observed surroundings to thepredetermined three-dimensional model using the knowledge that itscurrent position is little changed from its last computed position. Insome embodiments, the robotic platform 100 may utilize sensorsincluding, but not limited to, inertial measurement units (IMUs),odometers, and the like, to predict changes in location between LIDARscans.

With reference to FIG. 3, there is illustrated a view of the roboticplatform 100B showing the vacant cavity where a service module 102A-Dhas been removed according to an exemplary and non-limiting embodiment.On either side of the cavity there may be an affixed rail. The rails maybe adapted and positioned such that rollers attached to a service module102A-D may utilize the rails when sliding into position. Because of thesubstantial weight of the main body 214, there is generated aconsiderable force pushing down and out on the sides of the unit thattends to place stress on the main body. However, when a service module102A-D is inserted into the cavity, the mating of the main body 214 andservice module 102A-D provide tensile strength that serves to pull thetwo sides of the main body 214 together so as to counter act the forces,thus providing additional structural support to the robotic platform100. The cavity may include a locking mechanism, a power connection, asignal connection, and the like, to enable electrical connection betweenthe main body 214 and the service module 102A-D.

Service module 102A-D may include a locking mechanism, a signalconnection, a power connection, a tool controller, sliding mechanism,and the like. Each service module 102A-D may comprise a unique toolfunctional mechanism that may contain motors, actuators, brushes, fluidstorage, and the like. as appropriate to the service module function. Asdescribed herein, the service module 102A-D may likewise include one ormore sensors 106 associated with service module functioning such ascurrent sensors, pressure transducers, location sensors, and the like.

With reference to FIG. 4, there is illustrated a block diagram of thecomponents of a robotic platform 100 and the interaction between thecomponents of the main body 214 and a service module 102A-D. In general,each service module 102A-D has a computer control system to modify itsfunctions and its own software package. The service module 102A-D may beprogrammed for the function. In general, processes such as those relatedto navigation, localization, task management/initialization, and thelike, are performed using sensors and at least one processor 106A in themain body 214 while the service module 102A-D is programmed forapplication-specific functions. In some embodiments, a service module102A-D may comprise its own power source, such as a battery, so as notto burden the electrical load on the main body 214 of the roboticplatform 100. In some embodiments, service modules 102A-D, when not inoperation or attached to a main body 214, may be actively charging toprovide near continuous power availability upon demand. In someembodiments, charging may be performed in a contactless manner such asby, for example, inductive charging. In inductive charging the robotplatform 100 would monitor its state of charge and, when appropriate oras part of a pre-defined procedure, plan a route to a charging padlocated on the floor at some pre-designated location. Upon stopping overor near the charging pad the automated charging process could begin.

As illustrated, the main body 214 comprises user interface 402. The userinterface 402 may include all elements for a user to conduct taskplanning and to operate the equipment including, for example, visualinterface screen, element selection mechanism, on/off control, emergencystop and pause buttons, etc. The body 214 may further comprise a powerand battery management system 408 which may include battery cells andany required battery management systems. The body 214 may furthercomprise a charging system 406 to connect or interface with facilitypower to directly or inductively change the batteries. The body 214 mayfurther comprise a power distribution system 408 to provide electricalisolation and protection to the unit. Power distribution system 408 mayconvert and distribute battery power to voltages required by varioussystem elements. The body 214 may further comprise a processor 106A toexecute software to plan, manage, navigate and execute functions of thebody 214. The body 214 may further comprise a locomotion system 412 tocontrol the drive mechanism to move the body 214 and a feedback controlmechanism to implement guidance from the processing module 106A. Themain body 214 may further comprise a selection of various technologysensors 104A to sense the internal and external environment of the mainbody 214. It may include numerous sensor technologies such as inertial,vision, laser radar, ultrasonic, electromagnetic or other types ofsensor valuable for determining unit location, pose and condition. Themain body 214 may further comprise an external communications module 416to interface with external systems with radio frequency methods, directdigital methods, or audible methods.

Service module 102A-D may comprise a processor 106B to interface withthe main unit processing module 106A to receive, plan, control andexecute service module related tasks. Service module 102A-D may furthercomprise module sensors 104B to provide information related to servicemodule tasks include such elements as pressures, distances, visualconditions, solution levels, etc. Service module 102A-D may furthercomprise one or more end effector control elements 422 to initiate,monitor, and control end effector 424 elements such as motors,actuators, and other functions associated with the module functioningfor accomplishing the module's task, which may include a variety offunctions such as with brushes, cleaning or polishing heads, vacuumheads, manipulator arms, etc. Service module 102A-D may further includea power management module 426 to receive power from the main body 214 oran optional auxiliary power supply 428 to appropriately condition and/ordistribute it.

With reference to FIG. 5, there is illustrated an exemplary andnon-limiting embodiment of a robot localization process. As illustrated,various sensors may contribute to a layered processing of localization.For instance, an IMU 502 and rotary encoder 504 may provide inputs to anodometry and IMU extended Kalman filter 510, which along with othersensors, such as an ultra-wide band module 506 (e.g., utilizing signalscommunicated with RF locator nodes 150A-C) and a 2D laser scanner 508,are further provide input to a robot pose localization extended Kalmanfilter (global) 512. Additionally, the odometry and IMU extended Kalmanfilter 510 process may provide input, along with additional sensors suchas a camera 210, to an odometry filtered and visual odometry from cameraextended Kalman filter 516, which is then also provided as an input tothe robot pose localization extended Kalman filter (global) 512. Thislayered, sensor-based localization process may then contribute to thegeneration of a path planner node 514, such as in the generation of aplurality of path planner nodes providing robot localization as therobotic platform 100 moves through the service area in theimplementation of a service plan.

In other exemplary and non-limiting embodiments, the 3D LIDAR and/orstereoscopic imaging sensors may be used to perform pose estimation. Asused herein, “pose estimation” refers to localization with the addedattribute of orientation, such as a compass heading indicative of adirection the robotic platform 100 is facing or tilt. As a result, inmany instances, “localization” and “pose estimation” are usedinterchangeably.

With reference to FIG. 6, there is illustrated an exemplary andnon-limiting embodiment of a multi-sensor pose estimation process. Asillustrated, individual sensors 606A-D of various types that may includebut are not limited to inertial management units, a wheel odometer, 2and 3-dimensional laser ranging, ultrasonic sensors, electromagneticproximity detectors, radio frequency sensors, and the like, may compriseinputs to associated local smoothing filters 604A-D (analog or digital)to control signal and noise near the signal source. The outputs of thesmoothing filters may form inputs to associated unique sensor typeweighting and usage rules 602A-D forming a part of a pose estimationfilter 602D. The pose estimation filter may output an integrated poseestimate.

In accordance with exemplary and non-limiting embodiments, theultrasonic sensors 108 may be used to continually perform fineadjustments of the robotic platform's 100 location and pose to enablenear-continuous contact of tools forming a part of a service module102A-D with a floor and/or a wall intersection. In some embodiments,there may be a transition from a wall following mode of operation to abroader localization approach as described above. For example, when arobotic platform 100 approaches a wall, the ultrasonic sensors 106 maytake over and become the primary localization input.

In other embodiments, the 3D LIDAR and/or stereoscopic imaging sensorsmay be used to perform object detection and obstacle mitigation. Forexample, 3D LIDAR or stereoscopic imaging sensors may be used, alone orin conjunction with, for example, ultrasonic sensors 106, to determineif an object is obstructing the motion of the robotic platform 100 alongan intended path of travel. In such instances, the robotic platform maystop to avoid the object and may further engage in one or more of aplurality of mitigation techniques in a hierarchical manner. Forexample, if the object can be identified, such as by visual inspection,LIDAR scanning or the like, it may be determined that the object isstatic and may be moved. In such instances, the robotic platform mayemploy an articulated appendage to move the object. In other instances,the robotic platform may decide to transmit a communication, such as onecontaining a picture of an obstruction, to a supervisory person (e.g.,through a user computing device 130) who may respond with instructionsor may arrive on scene to remove the obstruction. In some instances, theobstacle may be dynamic, such as a dog or person, and may respond torequests to move out of the way. In some instances, the robotic platform100 may autonomously act to alter its intended path. In other instances,the robotic platform may cease operation if conditions dictate. Inembodiments, the robotic platform 100 may store instances of disruptedoperations for further processing, such as to come back to that locationlater to cover an area that was skipped.

In accordance with various exemplary embodiments, strategies to dealwith obstructions may depend upon a hierarchical decision-making processthat receives input data from the robotic platform's 100 varioussensors. Further, the robotic platform 100 may have the ability toupdate a predetermined three-dimensional map based upon the repeatedappearance of one or more obstacles 146. For example, a robotic platform100 used to polish the floors of an airport may proceed in accordancewith a model that defines a first configuration of seats at a boardinggate. When the seats are reconfigured in accordance with asemi-permanent or permanent new configuration, the seats may beinterpreted by the robotic platform 100 as comprising obstacles.However, over time, the robotic platform 100 may operate to ensure thereclassification of such obstacles as being enduring physical constructsforming, for example, a part of an updated three-dimensional model ofthe environment.

The process of object detection and mitigation may vary depending, atleast in part, upon the unique environment and conditions within which arobotic platform 100 is operating. For example, if a robotic platform100 is mowing a lawn and sees a child, it may stop, issue a warning, andproceed around the child. The speed of the robotic platform 100 (e.g.,dictated by a plan) and the service module 102A-D (e.g., dictated by theapplication) may result in varying danger levels. For example, aslow-moving floor waxer may be less dangerous than a fast-moving mower.As such, object detection and avoidance may be handled in a variety ofways described herein.

With reference to FIG. 7, there is illustrated an exemplary andnon-limiting embodiment of a sensing, obstacle avoidance, and pathplanning process. As illustrated, sensors, such as a side panelcapacitive sensor 702, wall following sensors 704, ultrasound sensors706, 2D LIDAR 708, rear segmented LIDAR 710, camera 210, and the like,may provide input sensor sources for obstacle avoidance 720 that maythen be provided to a global planner 722 for a plurality of planningalgorithms, such as for a point A to point B algorithm 730, fullcoverage algorithm 732, spot cleaning algorithm 734, wall followingalgorithm 736, and the like. Algorithmic outputs may then be used forsensing, obstacle avoidance, and path planning for the robotic platform100, such as when the robotic platform encounters an obstacle 146 thatis either anticipated or unexpected with respect to the service planbeing executed.

In accordance with various exemplary and non-limiting embodiments, theremay be employed a suite of tools to create, maintain, and attribute thethree-dimensional model of the environment. There are discussed abovevarious means by which the sensors on the robotic platform 100 may beemployed to generate such a model. As discussed, a point cloud may beobtained from a 3D LIDAR sensor. In other embodiments, a stereoscopicimaging system may be employed to create a 3D model. In such instances,one or more software modules may be employed to, for example, manipulatethe raw point cloud and/or further attribute the resulting model. Forexample, various areas forming a spatial part of the model may beattributed with multidimensional data. Such attributes may apply to anyof a variety of defined points, areas, and volumes. For example, using aGUI interface, a human operator or user may attribute areas of thethree- dimensional model depending on surface type. This point cloud maybe converted on the main unit or on an external processing system totwo-dimensional representations of the area. Alternatively, a spatialrepresentation such as a floor plan derived from the three-dimensionalrepresentation or separately derived can be used as the basis fordescribing surfaces types. In the representation, the user may bepresented with a floor-map like display. The user is then able toidentify work areas by defining area vertices. These defined work areasare then attributed with various information such as height of fixedobstacles or surface types. Examples of surface types include, but arenot limited to, marble, carpet, wood, grass, etc. Other attributes maybe temporal in nature. For example, areas that experience a high volumeof foot traffic may be designated to be cleaned nightly whereas otherless traveled areas may be designated to be cleaned weekly. Further,areas and points may be attributed to specify modes of operation thatare to be performed upon them and the order in which different modes areto be performed. For example, an area comprised of a wood floor may bedesignated to be swept with a service module 102A adapted to sweep priorto being polished by a service module 102B adapted to perform polishing.If developed on a separate processor, this attributed model may bedownloaded to the robotic platform 100 for operation.

Once attributed, the attributed model may be used as input to analgorithmic methodology for determining an efficient plan for traversingall required tasks. This methodology may consist of two parts, such asordering the tasks to be performed and then describing the specific pathplan to be taken to the different work areas and within the work areas.Once developed, the robotic platform 100 may commence to follow the planin an autonomous or semi-autonomous manner to complete the tasksspecified in the plan. Successful completion of the tasks may make useof on-board processing 106 of the robotic platform 100 to determine howbest to complete a task. For example, a robotic platform 100 may receivea plan calling for the vacuuming of a first area using a first servicemodule 102A and the buffing of a second area using a second servicemodule 102B. The plan may not specify the precise route for the roboticplatform 100 to follow to perform either task. Rather, the roboticplatform 100 may utilize an on-board processor to determine an optimizedpath to follow to complete each task. In such instances, the path to befollowed to exchange interchangeable service modules 112 may bedetermined on-board as well. As used herein, an “optimal” or “optimized”path is a path determined to be preferable based upon one or moreperformance metrics. In instances where information specifying a surfacetype is not provided to the robotic platform 100, the robotic platform100 may operate autonomously to detect a surface characteristic and (insome embodiments) select a service corresponding to the detectedcharacteristic. For example, an extendable spinning wheel may be used tomeasure mechanical resistance of a surface. In another example,reflectance patterns, including color and directional intensity, may besensed and used to identify a surface. In yet another example, an IRdetector may detect attributes of vegetation and soil levels including,but not limited to, vegetation height and moisture level. These detectedattributes might be used to determine, for example, a blade height or achemical to be applied. Based on the surface type identified, therobotic platform 100 may alter its mode of operation, including itschoice of service modules 112.

In some instances, a single service module 100 may, in response tosensing a surface characteristic, select a mode of operation and deployto service the sensed area in accordance with the selected service andutilizing the appropriate service module 102A-D.

In addition to the provided plan, the robotic platform 112 may make useof various static or dynamically updateable rules when performing tasks.With respect to the previous example, a rule may specify that whensweeping and buffing of single surface is indicated, the sweeping modeof operation should always be performed prior to the buffing mode ofoperation.

In some embodiments, the robotic platform 100 may react to priorityattributes or may impose priority based decision making in an autonomousfashion when completing tasks. For example, a plan may specificallyindicate which areas are to receive priority. In other instances, therobotic platform 100 may decide, for example in the instance where timeconsuming obstacle avoidance or mechanical failure has been encountered,that it is not possible to complete the specified plan in the amount oftime available to complete the tasks. In such instances, the roboticplatform 100 may make autonomous decisions regarding the completion anddeletion of various tasks or, conversely, may notify a human supervisorof an inability to complete the tasks as specified and may in responsereceive an updated plan.

In yet other exemplary embodiments, plans may be dynamically generatedbased on requirements. For example, a plan may be generated based on thedirective to scrub all areas that can be scrubbed per a specifiedservice plan. Such a directive may result in the production of a newplan to be transmitted to a robotic platform 100, or, conversely, thedirective itself may be transmitted to a robotic platform 100 thatproceeds to implement a current plan in accordance with the directive.Such as described, where it is apparent that a robotic platform 100 canavoid surfaces which are designated or attributed as not to be touched.

In accordance with other exemplary embodiments, the robotic platform 100may make use of external data to inform the mode of operation. Forexample, if the robotic platform 100 receives information that theweather prior to the performance of a nightly floor cleaning is stormyor rainy, the robotic platform 100 may adjust the speed at which itperforms sweeping of a floor area to take into the account the likelyincreased presence of dried dirt and mud. Likewise, external datasources for temperature readings and building management information maybe accessed and utilized.

Because, as envisioned, the robotic platform 100 will generally bemoving throughout a defined environment, the robotic platform may beemployed as a real-time reconnaissance apparatus. For example, therobotic platform 100 may, in addition to performing tasks specified inthe plan, perform security tasks. In some exemplary embodiments, suchtasks may be passive. For example, a robotic platform 100 may utilize acamera sensor and facial recognition software to note the presence ofpeople in the environment in which it operates. Such data may be storedfor later analysis or may be used to transmit alerts to human personnelin real time. Likewise, the robotic platform 100 may utilize any of itssensors to detect distinctive sounds such as the breaking of glass orthe sound of voices. In some instances, the robotic platform 100 mayrecord, attribute, such as with a time stamp, and store such noises forlater retrieval and use. In yet other embodiments, the robotic platform100 may use near-IR sensors, IR sensors, thermal imaging sensors and/orthermal sensors to observe and record temperature variations and to actin response to the sensing of same. For example, if a robotic platform100 senses the heat signature of footprints across a marble floor atmidnight in an otherwise empty airport terminal, such information may beused to generate and transmit an alert to security personnel. Likewise,a robotic platform 100 may alert a user or human operator to thepresence of smoke or fire. In other exemplary embodiments, the roboticplatform 100 may sense various environmental parameters, the functioningof an HVAC system, and structural anomalies.

While described with reference to a single robotic platform 100, inaccordance with exemplary and non-limiting embodiments, more than onerobotic platform 100 may act in concert to complete a plan. In suchinstances, robotic platforms 100 may be enabled to engage in directcommunication with one another or via an external server. For example, alarge environment such as an airport terminal may utilize multiplerobotic platforms 100 running an overall coordinated service plan. Bothrobotic platforms 100 may for instance be running with a similar type ofservice module 102A, such as a vacuum service module, in order toservice a large open space in a shorter amount of time. A first roboticplatform working on a first surface area may finish before a secondrobotic platform working on a second surface area, and may then beginworking on the second surface area to shorten the time for completingwork of vacuuming the total area. Alternately, the first roboticplatform may switch to a new service module 102B, such as a waxingservice module, and begin the process of waxing area that has alreadybeen vacuumed. Multiple robotic platforms may coordinate automatically,such as based on an established coordinated plan, or through a user,such as communicating through and being controlled by a user through auser computing device 130.

As described herein, a robotic platform 100 may be equipped with a 2DLIDAR system 208. In accordance with exemplary and non-limitingembodiments, such 2D LIDAR systems may be utilized for localization,pose estimation, navigation, surface detection and/or object/obstacledetection, mitigation, and avoidance. As illustrated in FIG. 1, the 2DLIDAR system 208 may be positioned or adapted to provide a forwardfacing cross section of the environment around the robotic platform 100and, generally, in the direction of travel. Because of the line of sightrequirement for LIDAR systems, the body of the robotic platform 100 maycause a blind spot for the LIDAR imaging sensor 204, such as immediatelyaround the robotic platform 100 and proceeding in a generally downwardconic section out, away and to the floor. The positioning of a 2D LIDARsystem 208 as illustrated and described allows for LIDAR, ultrasonic orvision-based detection of objects inside of the blind spot.

As described herein, a robotic platform may implement an algorithm tomatch the sensed surrounding environment to a predeterminedthree-dimensional model in order to determine a present location. Insome instances, it may prove difficult to perform such localizationusing such a methodology alone. For example, consider the case of a longhallway with identical doors positioned equidistant from one another.There may be various positions along such a hallway where thesurrounding three-dimensional environment appears identical or nearlyidentical. In such instances, and in accordance with exemplary andnon-limiting embodiments, there may be employed a network of two or moreultra-wide band RF locator nodes 150A-C to aid with localization, poseestimation and navigation. In some instances, software may be employedusing the three-dimensional model as input to deduce likely dead spotsin ultra-wide band coverage and to suggest optimal or near optimallocations for placing ultra-wide band signal transmitters.

Ultra-wide band signals are capable of passing through walls and othersolid barriers. Ultra-wide band RF locator nodes 150A-C may be placed atknown positions in an environment and may transmit identifying signals.A robotic platform 100 may exchange signals with one or more RF locatornodes 150A-C to determine distance measurements between the roboticplatform 100 and the anchors RF locator nodes 150A-C. Based, in part, onthe reception of two or more such transmissions, a robotic platform 100may apply trigonometry to determine a location relative to thetransmitters having absolute locations which are predetermined orotherwise known.

In the instance that four or more such ultra-wide band RF locator nodes150A-C are used, a robotic platform 100 may additionally determine aposition in a vertical or “z” direction in addition to the “x” and “y”horizontal coordinates. As described more fully herein, robotic platform100 may make use of a plurality of RF locator nodes 150A-C whenperforming localization. In such instances, the robotic platform 100 mayemploy a statistical process, such as Extended Kalman filtering, tomerge the location data in a manner that seeks to maximize the accuracyof the combined data readings. Such methods may assign weighting orgreater relevance to data sources known to produce more reliablereadings. Data sources may be weighted by inherent or recent conclusionsregarding sensor calibration or functionality, or applicability of thesensor technology in a particular operational environment. In othercases, weighting may reflect a predefined hierarchy based on theimmediate situation as perceived by the analysis of other sensor data.

In accordance with an exemplary and non-limiting embodiment, the roboticplatform may employ rules based decision-making process to determinewhen and if one or more location data sources should be used. Forexample, the robotic platform 100 may operate in accordance with staticor dynamically configurable default settings to determine when toincorporate different location data sources. For example, if, after acouple of seconds, the robotic platform 100 is unable to obtain a fix onits position using 3D LIDAR (or stereoscopic imaging sensors), it mayswitch to using ultra-wide band signals. The robotic platform 100 maysubsequently switch back to using 3D LIDAR when it is feasible to do so.For example, it may be the case that the use of 3D LIDAR yields threepossible locations for the robotic platform 100. The use of ultra-wideband signals may prove sufficient to resolve the ambiguity and therobotic platform 100 may proceed once again to use the 3D LIDAR forpurposes of localization.

As noted above, the robotic platform 100 may employ various rule-basedalgorithms to weight the relevance of differing sensors and to determinewhich sensors to use at any given moment. For example, it may bedetermined to weight, generally, the sonar data received from ultrasonicsensors 108 over the use of LIDAR. Another rule may indicate that, whenthe LIDAR is rendered deficient due, for example, to direct sunlightfalling on the sensor, the robotic platform 100 should switch to morereliance the use of ultra-wide band signals. Over time, such rules maybe altered via machine learning to more closely match the uniqueattributes of a specific environment. For example, a room that containsglass surfaces, such as aquariums, may negate the accurate operation of3D LIDAR. In such instances, tasks performed by the robotic platform 100in the performance of a plan that requires cleaning such a room mayproceed according to a rule that notes the preferred use of ultra-wideband signal localization when in the room. Alternatively, such a rulemay be embedded as an attribute of the area as specified in thethree-dimensional model of the environment.

As described herein, a robotic platform 100 may change its operationalcapabilities through reconfiguration. Specifically, each roboticplatform 100 is adapted such that individual service modules 102A-D maybe removed and inserted in an interchangeable fashion. As a result, itis not necessary to operate, for example, five separate roboticplatforms 100 in order to perform tasks comprising five different modesof operation, e.g., mowing, cutting, sweeping, polishing, and the like.Rather, the same robotic platform 100 may configure its operatingcapabilities by swapping one service module 102A-D for another, inaccordance with a service plan that is either determined by the roboticplatform 100 or otherwise communicated to it.

In an exemplary and non-limiting example of the wide variety offunctions that may be provided by a service module 102A-D utilized bythe robotic platform 100, a damp-wet mopping system 800 may be providedfor the cleaning of floors, such as depicted in FIG. 8. As illustrated,the mop surface is comprised of a roll of absorbent material stored on afirst roll 804 that may be pulled transverse across the width of moppingsystem 800 and stored on a second roll 802. As illustrated, as thesecond roll 802 stores used absorbent material it may increase in sizewhile the amount of absorbent material on first roll 804 decreases insize in complimentary fashion. In some embodiments, a predeterminedlength of the absorbent material is pulled across the width of thedamp-wet mopping system 800 and locked into place, such as at anagitator 808. Once locked, the absorbent material is placed in contactwith a surface to be cleaned. In some embodiments, the service moduleproviding the damp-wet mopping function may operate to agitate thematerial in a circular or linear motion as the robotic platform 100moves across the floor. After a predetermined amount of time or distanceis traveled across the floor, the section of the absorbent material incontact with the floor may be pulled through and stored on the secondroll 802 providing a clean length of absorbent material for use. Thedamp-wet mopping system 800 may comprise a cleaning fluid reservoir 806for storing cleaning fluid.

Continuing with the non-limiting damp-wet mopping system example, thefloor cleaning service module that comprises the damp-wet mopping systemmay be in communication with a user through a user interface on a usercomputing device 130, as described herein. Using the service moduledamp-wet mopping system as an example service module functionality, anon-limiting example of the user interface will be described, asillustrated in FIGS. 9-33, whose references will be made in theforegoing illustrative description. Through the user interface, anillustrative home screen 900 may provide the user with a view of aplurality of top-level user options, such as a ‘user’ area, an‘administrator’ area, a ‘service’ area, and the like. A user may be ableto select a name through a name screen 1000, such as to allow aplurality of users to interact through the user interface, such as togain access to robot platform functionality through a password screen1100. A function screen 1200 may provide for a listing of robotfunctions, such as for selecting a cleaning plan, for manually selectingtasks, selecting a destination, to establish manual control, to accesscurrent tool information, to access settings, and the like. A cleaningplan screen 1300 may provide for aspects of a plan, such as the title ofa plan, the tools required for a plan, and the like. For instance, acleaning plan may be for damp-wet mopping a floor, and the damp-wetmopping system 800 is listed as a required tool to be used. A listing ofplans screen 1400 may provide for the tool used, the work area, the taskarea, the approximate time required for the task, and the like. Forinstance, the work area may be a series of rooms, the task area is in ahospital, and the approximate time for damp-wet mopping the rooms is onehour. An add new plans screen 1500 may be provided, such as listing thenew plan, the tool to be used, the task to be performed, a listing ofthe plan, an ‘add’ button for adding the new plan, and the like. A homelocation screen 1600 may be provided that indicates, for instance, wherethe robot platform would start mopping the floor for the task area. Inaddition, control and/or indicators of various functions and tasks maybe provided, such as for manual control of the robotic platform, for ahome location being reached, and the like.

Once a task has been initiated, a status-control screen 1700 may beprovided, such as providing current status information (e.g.,approximate time for task completion, location, tool, and the like).Function control buttons may be provided, such as to cancel or continuea current task. A time-down screen 1800 may be provided, such as forwhen a cleaning is set to begin, an approximate time for the task to becompeted, and the like, and where a cancel function button may beavailable. A progress screen 1900 may be provided, such as with a visualor percentage indication as to how far along the task has progressed. Aservice plan progress screen 2000 may be provided, such as illustratingthe progress complete in the current task, the progress complete for aplurality of tasks, and the like. The robot platform 100 may indicate tothe user that a tool needs to be changed, such as through a tool changerequired screen 2100. For instance, the robot platform 100 may havecompleted the damp-wet mopping task and needs to transition to a waxingtask. In this instance, a first service module 102A may need to beswitched for a second service module 102B. The need for switching toolsmay be indicated on an alert screen 2200, 2300, which may provide theuser with the options to cancel cleaning, direct the robotic platform100 to proceed to a service module exchange facility 114, and the like.

A completion screen 2400 may be provided once the task or service planis complete, such as providing a completion indication, that tasks havebeen complete, and the like, with options for viewing task completionoptions (e.g., viewing uncleaned locations, to conclude a task or plan).A completion screen 2500 may illustrate that the robotic platform 100 isreturning to its home station. Once a task or plan has been completed, amap view illustration screen 2600 may be provided to indicate areas therobot platform 100 missed, such as skipping as a result of encounteringan obstacle 146, needing to return to a special treatment area 144, andthe like. A manual tasking screen 2700 may be provided, such as tospecify a new task to be added. In embodiments, the task may be added tothe current tasking for the day, to the service plan as a new task forsubsequent days, and the like. A select destination screen 2800 may beprovided, such as to send the robotic platform 100 to a specifiedlocation, floor, building, and the like. A plurality of screens may beprovided to indicate status of the location change, such as adestination list screen 2900 provided to help identify the location tosend the robotic platform 100, a selected location execution statusscreen 3000 to enable pausing or canceling the function, a systemarrived screen 3100 to indicate completion of the location transfer, andthe like. Further, a manual control screen 3200 may be provided toinitiate or cancel manual control of the robotic platform 100. Withrespect to the tool currently being used, there may be a current toolscreen 3300, such as showing an image of the tool, status for toolheads, hours run, command buttons for attending the service module(e.g., need to change a bag or a consumable fluid, changing a cleaninghead), and the like. For example, in the case where the damp-wet moppingsystem 800 is the tool being currently utilized, an image of the toolmay be provided along with status indications, such as with respect tothe absorbent material and cleaning fluid used.

The present disclosure describes methods and systems for areconfigurable robotic platform 100 utilizing a plurality ofinterchangeable service modules 102A-D and adapted to engage in bothautonomous and interactive maintenance and monitoring of a service area.The robotic platform 100 may be configured to perform a wide variety oftasks utilizing the plurality of interchangeable service modules, andnavigating through the service area utilizing a plurality of sensors andguided through a stored service plan for the service area. Variousembodiments of the robotic platform 100 will now be described.

In embodiments, the robotic platform 100 may provide for service plannavigation utilizing a plurality of RF transmitting location nodes, suchas implementing navigation of a service plan 128 through service areas140A-B utilizing a plurality of RF locator nodes 150A-C. RF locatornodes 150A-C may be utilized in navigation alone or in combination withother navigation sensors 104, such as 3D LIDAR or stereoscopic imagingsensors, especially when such sensors are constrained in some way by thesurrounding environment. As described herein, RF locator nodes are notessential to navigation. Other methods such as vision or LIDAR may beemployed so long as ambiguity of position can be resolved throughon-board sensors 104. In the instance when sensors 104 cannot adequatelyresolve positional ambiguity, RF locator nodes 150A-C may be utilized toprovide a reference grid for resolving the lack of knowledge of positionambiguity. In embodiments, the robotic platform 100 may comprise one ofa plurality of removable service modules 102A-D and a propulsionmechanism 202 to position the robotic platform 100 at a plurality ofpredetermined locations, wherein the predetermined positions are locatedbased, at least on part, on one or more transmitting RF locator nodes150A-C, where the robotic platform 100 operates the removable servicemodule to service a generally planar surface proximate the robot.

In embodiments, the robotic platform 100 may provide for location andsensor-based detection of a transition between separately definedservice areas 140A-B in a service plan 128, such as automaticallydetecting a surface change (e.g., rug to tile) at a location anddetermining that a tool change is required to transition from oneservice area 140A to another 140B as determined from the service plan128. In embodiments, the robotic platform 100 may sense an environmentaround a robotic platform 100 comprising a removable service module102A-D, operating the removable service module to service a generallyplanar surface proximate the robotic platform 100 in accordance with aservice plan 128, sensing a need to change the removable service module102A-D in response to a change in a type of the generally planar surfaceproximate the robotic platform 100, such as where the change is definedin a service plan.

In embodiments, the robotic platform 100 may provide for sensor-baseddetection and tracking of a service-area-edge-condition during servicearea plan 128 execution, such as detecting and tracking a wall or edgeof a floor type during a service plan execution. In embodiments, therobotic platform 100 comprising a removable service module 102A-D maysense a surrounding environment, and operate the removable servicemodule 102A-D to service a generally planar surface proximate therobotic platform 100 in accordance with a service plan 128 where sensingthe environment comprises sensing at least one edge of at least oneservice area 140A-B defined in the service plan 128.

In embodiments, the robotic platform 100 may provide for sensor-baseddetection of a service event condition within a single defined servicearea 140A, such as finding a location within an area that requiresspecial attention beyond the routine service plan (e.g., a specialtreatment area 144) but for which there may exist an action protocol(e.g., spot treatment of the area). In embodiments, the robotic platform100 may sense an environment around a robotic platform 100 comprising aremovable service module 102A-D, and operate the removable servicemodule 102A-D to service a generally planar surface proximate therobotic platform 100 in accordance with a service plan 128, and deviatefrom the service plan 128 in the event that an anomalous condition ofthe planar surface is detected.

In embodiments, the robotic platform 100 may provide for sensor-baseddisruption detection and avoidance during execution of a service plan128 in a service area 140A, such as the when the robotic platform 100encounters an unplanned obstacle 146 (e.g., a chair out of place),avoids it, and continues service. In embodiments, the robotic platform100 comprising a removable service module 102A-D may sense a surroundingenvironment, and operate the removable service module 102A-D to servicea generally planar surface proximate the robotic platform 100 inaccordance with a service plan 128, and deviate from the service plan128 in the event that an obstacle 146 to the servicing of the planarsurface is detected. The robotic platform 100 may store the location ofthe obstacle 146 as a location to return to in subsequent service.

In embodiments, the robotic platform 100 may provide for operationalservice plan disruption and return optimization for a service roboticplatform 100, such as when the robotic platform 100 encounters anobstacle 146 that forces it to skip a portion of a task, and establishesa return plan for subsequently returning (e.g., based on a calculatedcost-trade for the obstacle relative to the overall service plan). Inembodiments, the robotic platform 100 comprising a removable servicemodule 102A-D may sense a surrounding environment, operate the removableservice module 102A-D to service a generally planar surface proximatethe robotic platform 100 in accordance with a service plan 128, deviatefrom the service plan 128 so as to suspend a performance of a taskdefined in the service plan 128, produce an updated service plancomprising at least one task directed to completing the suspended task,and operate the removable service module 102A-D to service the generallyplanar surface proximate the robotic platform 100 in accordance with theupdated service plan.

In embodiments, the robotic platform 100 may provide for an alerttransmission of a detected service-module-change-condition to a wirelessuser interface, such as alerting an on-site user with a user computingdevice 130 that the robot needs to have a service module 102A-D manuallychanged out (e.g., at a service module exchange facility 114). Inembodiments, the robotic platform 100 comprising a removable servicemodule 102A-D adapted to perform a predefined service task may receivean alert, where the removable service module 102A-D is manually replacedin response, at least in part, to the alert.

In embodiments, the robotic platform 100 may provide for automaticreplacement of a service module 102A-D in a service module dockingstation (e.g., service robot resource facility 110), such as when therobotic platform 100 automatically replaces the service module 102A-Dwith limited user assistance. In embodiments, the robotic platform 100comprising a removable service module 102A-D adapted to perform apredefined service task may receive an alert, and automatically replacethe removable service module 102A-D in response, at least in part, tothe alert.

In embodiments, the robotic platform 100 may provide for operativeservice area planning for a robotic platform 100 through digital mapannotation, such as providing manual service planning through annotatinga digital map that has been loaded into the robotic platform 100.Digital maps, as described herein, may be 2D or 3D digital maps,generated in conjunction with operation of the robotic platform 100 ordownloaded to the robotic platform from an external computing device(e.g., when the digital map was generated externally). In embodiments,the robotic platform 100 comprising a removable service module mayreceive at a service plan 128 comprising instructions for servicing agenerally planar surface where the service plan 128 is derived, at leastin part, from an annotated map, and operate the robot to service thegenerally planar surface.

In embodiments, the robotic platform 100 may provide for identificationof a surface type through a service area annotation in a stored digitalmap, such as annotating surface type through the stored digital mapduring service planning (e.g., the surface type is an indoor-outdoorrug, a linoleum tile, thick carpet). In embodiments, the roboticplatform 100 comprising a removable service module may receive at aservice plan 128 comprising instructions for servicing a generallyplanar surface where the service plan 128 is derived, at least in part,from an annotated map specifying a surface type, and operate the robotto service the generally planar surface.

In embodiments, the robotic platform 100 may provide for multi-operativeservice area planning through a digital map utility, such as multi-areaplanning of different service areas 140A-B using a digital map loadedinto the robotic platform 100. In embodiments, the robotic platform 100comprising a removable service module may receive a service plan 128comprising instructions for servicing a plurality of generally planarsurfaces, and operate the robotic platform 100 to service the generallyplanar surfaces.

In embodiments, the robotic platform 100 may provide for digital maputilization in the automatic execution of a service plan 128, such asthe robotic platform 100 using the digital map to execute a storedservice plan 128. In embodiments, the robotic platform 100 comprising aremovable service module may receive a service plan 128 comprisinginstructions for servicing a generally planar surface of a service area140A-B, and utilize a digital map of the service area 140A-B to operatethe robotic platform 100 to service the generally planar surface.

In embodiments, the robotic platform 100 may provide for real-timeservice area plan identification through utilization of a stored digitalmap and surface type sensors, such as the robotic platform 100determining (e.g., in real-time) an execution plan in a service area140A-B guided by a digital map and utilizing sensors 104 to detect asurface type and required service tool. In embodiments, the roboticplatform 100 comprising a removable service module may sense asurrounding environment, derive a service plan 128 for servicing theenvironment based, at least in part, on the sensed environment and adigital map of the environment, and operating the removable servicemodule 102A-D to service a generally planar surface proximate therobotic platform 100 in accordance with the service plan 128.

In embodiments, the robotic platform 100 may provide for automaticservice area mapping by a robotic platform 100, such as where therobotic platform 100 maps service areas 140A-B to produce a service map.The robot mapping may be an autonomous process, where the roboticplatform 100 maps the service area(s) 140A-B by exploring the area onits own, or the robot mapping may be performed in conjunction with aidfrom a user, such as through a user operating the robotic platform 100through manual control. In embodiments, the robotic platform 100comprising a removable service module 102A-D may sense a surroundingenvironment, and produce a service map of the environment based, atleast in part, on the sensed environment.

In embodiments, the robotic platform 100 may provide for sensor-basedadaptive service plan tasking, such as adjustment of a service plan taskexecution based on environment changes (e.g., unexpected obstaclesand/or human presence). In embodiments, the robotic platform 100comprising a removable service module 102A-D may sense a surroundingenvironment, operate the removable service module 102A-D to service agenerally planar surface proximate the robotic platform 100 inaccordance with a service plan, and deviate from the service plan so asto suspend a performance of a task defined in the service plan inresponse to a detected change in the sensed environment, producing anupdated service plan comprising at least one task directed to completingthe suspended task, and operating the removable service module 102A-D toservice the generally planar surface proximate the robotic platform 100in accordance with the updated service plan.

In embodiments, the robotic platform 100 may provide for machinelearning adaptive service plan tasking, such as the robotic platform 100adjusting its plan tasking (e.g., from day to day) based on what it haslearned from past service executions. In embodiments, the roboticplatform 100 comprising a removable service module may sense asurrounding environment, and operate the removable service module 102A-Dto service a generally planar surface proximate the robotic platform 100in accordance with a predetermined service plan 128, where the serviceplan 128 incorporates information gathered by the robotic platform 100during past operations of the service module 102A-D.

In embodiments, the robotic platform 100 may provide for operationalsensor suite optimization selection for power conservation in therobotic platform, such as for conserving power through selection ofactive sensors during service. In embodiments, the robotic platform 100comprising a removable service module 102A-D may sense a surroundingenvironment using one or more active sensors 104 around the roboticplatform 100, and operate the removable service module 102A-D to servicea generally planar surface proximate the robotic platform 100 inaccordance with a service plan 128 while selectively activating at leastone of the active sensors 104. Selection of active sensors 104 may alsoprovide optimized navigation performance, adjustment to environmentalconditions, and the like.

In embodiments, the robotic platform 100 may provide for operationalsensor suite optimization selection based on changes in environmentalconditions, such as changing the sensors 104 to improve performance inthe presence of a changed environment (e.g., the environment may requirean increased resolution from the sensor suite, such as for increasedhuman presence, or the presence of a detected plurality of obstacles146). In embodiments, the robotic platform 100 comprising a removableservice module 102A-D may sense an environment using one or more activesensors 104 around the robotic platform 102, and operate the removableservice module 102A-D to service a generally planar surface proximatethe robotic platform 100 in accordance with a service plan 128 whileselectively changing at least one of the active sensors 104 based on asensed change in the environment.

In embodiments, the robotic platform 100 may provide for placementoptimization of transmitting RF location nodes 150A-C for service plannavigation, such as optimum placement of RF location nodes 150A-C priorto service plan execution. In embodiments, the robotic platform 100comprising a removable service module may determine a location for eachof a plurality of RF transmitting location nodes, and positioning therobotic platform 100 at a predetermined location wherein thepredetermined location is located based, at least on part, on the one ormore RF transmitting RF location nodes 150A-C.

In embodiments, the robotic platform 100 may provide for service robotnavigation through a service area location grid established through aplurality of transmitting RF location nodes 150A-C, such as fornavigating through a grid established through RF location nodes 150A-C.In embodiments, the robotic platform 100 comprising a removable servicemodule 102A-D may engage a propulsion mechanism 202 of the roboticplatform 100 to position the robotic platform 100 at a plurality ofpredetermined locations within a service area 140A-B, where thepredetermined positions are located based, at least on part, on a gridof established by a plurality of RF locator nodes 150A-C, and operatingthe removable service module 102A-D to service a generally planarsurface proximate the robotic platform 100.

In embodiments, the robotic platform 100 may provide for service robotnavigation during external navigation source disruption, such asproviding for robot platform navigation through dead reckoning when RFlocator node signals are temporarily unavailable (e.g., dead spot). Inembodiments, the robotic platform 100 comprising a removable servicemodule 102A-D may engage a propulsion mechanism to position the roboticplatform 100 at a plurality of predetermined locations using deadreckoning, and operating the removable service module 102A-D to servicea generally planar surface proximate the robot.

In embodiments, the robotic platform 100 may provide for coordination ofa plurality of service robots operating in proximity to one another,such as where multiple coordinated service robots act together toexecute a service plan 128. In embodiments, the robotic platform 100comprising a removable service module 102A-D may engage a propulsionmechanism to position the robotic platform 100 at a predeterminedlocation for operating the removable service module 102A-D to service agenerally planar surface proximate the robotic platform 100 inaccordance with a service plan 128, and receiving information from atleast one other robotic platform 100 operating in accordance with theservice plan to facilitate cooperative completion of the service plan.

In embodiments, the robotic platform 100 may provide for shared modulartool utilization amongst a plurality of service robots, such as for whenmultiple robotic platforms are using the same service modules 102A-D. Inembodiments, the robotic platform 100 comprising a removable servicemodule 102A-D may engage a propulsion mechanism to position the roboticplatform 100 at a predetermined location for operating the removableservice module 102A-D to service a generally planar surface proximatethe robot in accordance with a service plan 128, receive informationfrom at least one other robotic platform 100 operating in accordancewith the service plan 128 to facilitate cooperative completion of theservice plan 128, and in response to the received information, providingthe removable service module 102A-D to the other robotic platform 100.

In embodiments, the robotic platform 100 may provide for service robotpose determination through service area location grid establishedthrough a plurality of RF transmitting RF location nodes 150A-C, such asfor determining orientation of robot from a grid established through RFlocation nodes 150A-C. In embodiments, the robotic platform 100comprising a removable service module 102A-D may engage a propulsionmechanism to position the robotic platform 100 at a plurality ofpredetermined location wherein the predetermined positions are locatedbased, at least on part, on a grid of one or more transmitting RFlocation nodes, and operating the removable service module 102A-D toservice a generally planar surface proximate the robotic platform 100.

The programmed methods and/or instructions described herein may bedeployed in part or in whole through a machine that executes computersoftware, program codes, and/or instructions on a processor orprocessors. “Processor” used herein is synonymous with the plural“processors” and the two terms may be used interchangeably unlesscontext clearly indicates otherwise. The processor may be part of aserver, client, network infrastructure, mobile computing platform,stationary computing platform, or other computing platform. A processormay be any kind of computational or processing device capable ofexecuting program instructions, codes, binary instructions and the like.The processor may be or include a signal processor, digital processor,embedded processor, microprocessor or any variant such as a co-processor(math co-processor, graphic co-processor, communication co-processor andthe like) and the like that may directly or indirectly facilitateexecution of program code or program instructions stored thereon. Inaddition, the processor may enable execution of multiple programs,threads, and codes. The threads may be executed simultaneously toenhance the performance of the processor and to facilitate simultaneousoperations of the application. By way of implementation, methods,program codes, program instructions and the like described herein may beimplemented in one or more thread. The thread may spawn other threadsthat may have assigned priorities associated with them; the processormay execute these threads based on priority or any other order based oninstructions provided in the program code. The processor may includememory that stores methods, codes, instructions and programs asdescribed herein and elsewhere. The processor may access a storagemedium through an interface that may store methods, codes, andinstructions as described herein and elsewhere. The storage mediumassociated with the processor for storing methods, programs, codes,program instructions or other type of instructions capable of beingexecuted by the computing or processing device may include but may notbe limited to one or more of a CD-ROM, DVD, memory, hard disk, flashdrive, RAM, ROM, cache and the like.

A processor may include one or more cores that may enhance speed andperformance of a multiprocessor. In embodiments, the process may be adual core processor, quad core processors, other chip-levelmultiprocessor and the like that combine two or more independent cores(called a die).

The methods and systems described herein may be deployed in part or inwhole through a machine that executes computer software on a server,client, firewall, gateway, hub, router, or other such computer and/ornetworking hardware. The software program may be associated with aserver that may include a file server, print server, domain server,Internet server, intranet server and other variants such as secondaryserver, host server, distributed server and the like. The server mayinclude one or more of memories, processors, computer readable media,storage media, ports (physical and virtual), communication devices, andinterfaces capable of accessing other servers, clients, machines, anddevices through a wired or a wireless medium, and the like. The methods,programs or codes as described herein and elsewhere may be executed bythe server. In addition, other devices required for execution of methodsas described in this application may be considered as a part of theinfrastructure associated with the server.

The server may provide an interface to other devices including, withoutlimitation, clients, other servers, printers, database servers, printservers, file servers, communication servers, distributed servers andthe like. Additionally, this coupling and/or connection may facilitateremote execution of program across the network. The networking of someor all of these devices may facilitate parallel processing of a programor method at one or more location without deviating from the scope. Inaddition, any of the devices attached to the server through an interfacemay include at least one storage medium capable of storing methods,programs, code and/or instructions. A central repository may provideprogram instructions to be executed on different devices. In thisimplementation, the remote repository may act as a storage medium forprogram code, instructions, and programs.

The software program may be associated with a client that may include afile client, print client, domain client, Internet client, intranetclient and other variants such as secondary client, host client,distributed client and the like. The client may include one or more ofmemories, processors, computer readable media, storage media, ports(physical and virtual), communication devices, and interfaces capable ofaccessing other clients, servers, machines, and devices through a wiredor a wireless medium, and the like. The methods, programs or codes asdescribed herein and elsewhere may be executed by the client. Inaddition, other devices required for execution of methods as describedin this application may be considered as a part of the infrastructureassociated with the client.

The client may provide an interface to other devices including, withoutlimitation, servers, other clients, printers, database servers, printservers, file servers, communication servers, distributed servers andthe like. Additionally, this coupling and/or connection may facilitateremote execution of program across the network. The networking of someor all of these devices may facilitate parallel processing of a programor method at one or more location without deviating from the scope. Inaddition, any of the devices attached to the client through an interfacemay include at least one storage medium capable of storing methods,programs, applications, code and/or instructions. A central repositorymay provide program instructions to be executed on different devices. Inthis implementation, the remote repository may act as a storage mediumfor program code, instructions, and programs.

The methods and systems described herein may be deployed in part or inwhole through network infrastructures. The network infrastructure mayinclude elements such as computing devices, servers, routers, hubs,firewalls, clients, personal computers, communication devices, routingdevices and other active and passive devices, modules and/or componentsas known in the art. The computing and/or non-computing device(s)associated with the network infrastructure may include, apart from othercomponents, a storage medium such as flash memory, buffer, stack, RAM,ROM and the like. The processes, methods, program codes, instructionsdescribed herein and elsewhere may be executed by one or more of thenetwork infrastructural elements.

The methods, program codes, and instructions described herein andelsewhere may be implemented on a cellular network having multiplecells. The cellular network may either be frequency division multipleaccess (FDMA) network or code division multiple access (CDMA) network.The cellular network may include mobile devices, cell sites, basestations, repeaters, antennas, towers, and the like. The cell networkmay be a GSM, GPRS, 3G, 4G, LTE, EVDO, mesh, or other networks types.

The methods, programs codes, and instructions described herein andelsewhere may be implemented on or through mobile devices. The mobiledevices may include navigation devices, cell phones, mobile phones,mobile personal digital assistants, laptops, palmtops, netbooks, pagers,electronic books readers, music players and the like. These devices mayinclude, apart from other components, a storage medium such as a flashmemory, buffer, RAM, ROM and one or more computing devices. Thecomputing devices associated with mobile devices may be enabled toexecute program codes, methods, and instructions stored thereon.Alternatively, the mobile devices may be configured to executeinstructions in collaboration with other devices. The mobile devices maycommunicate with base stations interfaced with servers and configured toexecute program codes. The mobile devices may communicate on a peer topeer network, mesh network, or other communications network. The programcode may be stored on the storage medium associated with the server andexecuted by a computing device embedded within the server. The basestation may include a computing device and a storage medium. The storagedevice may store program codes and instructions executed by thecomputing devices associated with the base station.

The computer software, program codes, and/or instructions may be storedand/or accessed on machine readable media that may include: computercomponents, devices, and recording media that retain digital data usedfor computing for some interval of time; semiconductor storage known asrandom access memory (RAM); mass storage typically for more permanentstorage, such as optical discs, forms of magnetic storage like harddisks, tapes, drums, cards and other types; processor registers, cachememory, volatile memory, non-volatile memory; optical storage such asCD, DVD; removable media such as flash memory (e.g. USB sticks or keys),floppy disks, magnetic tape, paper tape, punch cards, standalone RAMdisks, Zip drives, removable mass storage, off-line, and the like; othercomputer memory such as dynamic memory, static memory, read/writestorage, mutable storage, read only, random access, sequential access,location addressable, file addressable, content addressable, networkattached storage, storage area network, bar codes, magnetic ink, and thelike.

The methods and systems described herein may transform physical and/oror intangible items from one state to another. The methods and systemsdescribed herein may also transform data representing physical and/orintangible items from one state to another.

The elements described and depicted herein, including in flow charts andblock diagrams throughout the figures, imply logical boundaries betweenthe elements. However, according to software or hardware engineeringpractices, the depicted elements and the functions thereof may beimplemented on machines through computer executable media having aprocessor capable of executing program instructions stored thereon as amonolithic software structure, as standalone software modules, or asmodules that employ external routines, code, services, and so forth, orany combination of these, and all such implementations may be within thescope of the present disclosure. Examples of such machines may include,but may not be limited to, personal digital assistants, laptops,personal computers, mobile phones, other handheld computing devices,medical equipment, wired or wireless communication devices, transducers,chips, calculators, satellites, tablet PCs, electronic books, gadgets,electronic devices, devices having artificial intelligence, computingdevices, networking equipment, servers, routers and the like.Furthermore, the elements depicted in the flow chart and block diagramsor any other logical component may be implemented on a machine capableof executing program instructions. Thus, while the foregoing drawingsand descriptions set forth functional aspects of the disclosed systems,no particular arrangement of software for implementing these functionalaspects should be inferred from these descriptions unless explicitlystated or otherwise clear from the context. Similarly, it may beappreciated that the various steps identified and described above may bevaried, and that the order of steps may be adapted to particularapplications of the techniques disclosed herein. All such variations andmodifications are intended to fall within the scope of this disclosure.As such, the depiction and/or description of an order for various stepsshould not be understood to require a particular order of execution forthose steps, unless required by a particular application, or explicitlystated or otherwise clear from the context.

The methods and/or processes described above, and steps thereof, may berealized in hardware, software or any combination of hardware andsoftware suitable for a particular application. The hardware may includea general-purpose computer and/or dedicated computing device or specificcomputing device or particular aspect or component of a specificcomputing device. The processes may be realized in one or moremicroprocessors, microcontrollers, embedded microcontrollers,programmable digital signal processors or other programmable device,along with internal and/or external memory. The processes may also, orinstead, be embodied in an application specific integrated circuit, aprogrammable gate array, programmable array logic, or any other deviceor combination of devices that may be configured to process electronicsignals. It may further be appreciated that one or more of the processesmay be realized as a computer executable code capable of being executedon a machine-readable medium.

The computer executable code may be created using a structuredprogramming language such as C, an object oriented programming languagesuch as C++, or any other high-level or low-level programming language(including assembly languages, hardware description languages, anddatabase programming languages and technologies) that may be stored,compiled or interpreted to run on one of the above devices, as well asheterogeneous combinations of processors, processor architectures, orcombinations of different hardware and software, or any other machinecapable of executing program instructions.

Thus, in one aspect, each method described above and combinationsthereof may be embodied in computer executable code that, when executingon one or more computing devices, performs the steps thereof. In anotheraspect, the methods may be embodied in systems that perform the stepsthereof, and may be distributed across devices in a number of ways, orall of the functionality may be integrated into a dedicated, standalonedevice or other hardware. In another aspect, the means for performingthe steps associated with the processes described above may include anyof the hardware and/or software described above. All such permutationsand combinations are intended to fall within the scope of the presentdisclosure.

While the methods and systems described herein have been disclosed inconnection with certain preferred embodiments shown and described indetail, various modifications and improvements thereon may becomereadily apparent to those skilled in the art. Accordingly, the spiritand scope of the methods and systems described herein is not to belimited by the foregoing examples, but is to be understood in thebroadest sense allowable by law.

Additional details of exemplary and non-limiting embodiments arerecounted in Appendix A attached hereto and filed herewith.

All documents referenced herein are hereby incorporated by reference.

What is claimed is:
 1. A robotic device comprising: a propulsionmechanism to move the robotic device; at least one position sensoradapted for tracking a location of the robotic device as it movesthrough a service area; a surface sensor adapted to monitor a surface ofthe service area for a cleanliness-based surface characteristic as partof a service task, wherein the service task is performed as the roboticdevice moves through the service area; a processing facility comprisinga processor and a memory, the processing facility storing a set ofinstructions that, when executed, cause the robotic device to: execute astored service plan for a service area, the service plan comprising aservice plan sequence comprising a service treatment to a surface of theservice area, identify a special treatment area utilizing the surfacesensor, alter the service plan sequence based on identifying the specialtreatment area in the service area, wherein the altering of the serviceplan sequence includes providing an additional service to the specialtreatment area, and resume execution of the service plan.
 2. The roboticdevice of claim 1, wherein the surface sensor is an imaging sensor forsensing anomalous stains.
 3. The robotic device of claim 1, wherein thecleanliness-based surface characteristic is one of a stain on a floorand a high level of soiling on the floor.
 4. The robotic device of claim1, wherein the cleanliness-based surface characteristic is a user-markedor spot-treated area of a floor.
 5. The robotic device of claim 1,wherein the service plan is a floor cleaning plan for a room and theservice plan sequence is a sequence of cleaning steps to clean the room.6. The robotic device of claim 5, wherein the special treatment area isan area of the floor that requires additional cleaning, and the set ofinstructions that, when executed, cause the robotic device to apply afurther cleaning to the special treatment area as part of the additionalservice.
 7. The robotic device of claim 6, wherein the further cleaningcomprises applying more pressure to a cleaning tool.
 8. The roboticdevice of claim 6, wherein the further cleaning comprises slowing down aspeed of the robotic device to increase an amount of service treatmentprovided.
 9. The robotic device of claim 6, wherein the further cleaningcomprises going over the special treatment area a second time.
 10. Therobotic device of claim 1, wherein the service treatment is one ofvacuuming or washing.
 11. The robotic device of claim 1, the roboticdevice further comprising a main robotic device portion and a servicemodule portion, the service module portion adapted to perform theservice plan, wherein the service module portion is removable from themain robotic device portion.
 12. A method comprising: executing a storedservice plan for a service area with a robotic device, the service plancomprising a service plan sequence comprising a service treatment to asurface of the service area, wherein the robotic device comprises apropulsion mechanism, at least one position sensor adapted for trackinga location of the robotic device as it moves through a service area, anda surface sensor adapted to monitor a surface of the service area for acleanliness-based surface characteristic as part of a service task;identifying a special treatment area utilizing the surface sensor;altering the service plan sequence based on identifying the specialtreatment area in the service area, wherein the altering of the serviceplan sequence includes providing an additional service to the specialtreatment area; and resuming execution of the service plan.
 13. Themethod of claim 12, wherein the surface sensor is an imaging sensor forsensing anomalous stains.
 14. The method of claim 12, wherein thecleanliness-based surface characteristic is one of a stain on a floorand a high level of soiling on the floor.
 15. The method of claim 12,wherein the cleanliness-based surface characteristic is a user-marked orspot-treated area of a floor.
 16. The method of claim 12, wherein theservice plan is a floor cleaning plan for a room and the service plansequence is a sequence of cleaning steps to clean the room.
 17. Themethod of claim 16, wherein the special treatment area is an area of thefloor that requires additional cleaning, and the robotic device appliesa further cleaning to the special treatment area as part of theadditional service.
 18. The method of claim 17, wherein the furthercleaning comprises applying more pressure to a cleaning tool.
 19. Themethod of claim 17, wherein the further cleaning comprises slowing downa speed of the robotic device to increase an amount of service treatmentprovided.
 20. The method of claim 17, wherein the further cleaningcomprises going over the special treatment area a second time.
 21. Themethod of claim 12, wherein the service treatment is one of vacuuming orwashing.
 22. The method of claim 12, the robotic device furthercomprising a main robotic device portion and a service module portion,the service module portion adapted to perform the service plan, whereinthe service module portion is removable from the main robotic deviceportion.